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DETAILED ACTION 

Response to Amendment 

1. This office action is in response to communications filed 5/21/2010. Claims 1-2 are 
amended. Claims 4-6 and 19 are cancelled. Claim 21 is new. Claims 1-3, 7-18 and 20-21 are 
pending in this action. 

Response to Arguments 

2. Applicant's arguments with respect to claims 1-4 and 6-20 have been considered but are 
moot in view of the new ground(s) of rejection. 

Although new grounds of rejection have been presented, a response to Applicant's 
arguments is still considered necessary since the Lemmons, Bulman and Kumar references, all of 
record, will continue to be used to meet several claimed limitations. 

In response to Applicant's arguments on page 9, line 17 - page 10, line 7 that, "However, 
the mere recording of orientation and position does not imply or require an index. The cited 
portions of Bulman, for example col. 13, lines 33-37, merely disclose matching images of a 
human head subject to a background image. No index is identified or transmitted as required by 
claim 12. 
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During the interview, the Examiner stated that the names Fl, F2, etc., as shown by Fig. 
11 may be considered an index because the names are associated with the various faces and 
appear to be an ordered list. Applicants respectfully disagree for two reasons. 

First, Fig. 11 including names Fl, F2, etc. is part of Example 5. See col. 10, line 64 - col. 
12, line 12. Example 5 does not involve different orientations. Instead, Example 5 involves a 
scanned photograph, see col. 11, lines 2-5, which is "automatically scaled up to 30 different 
sizes, from small to large, each of which is saved as an individual foreground image with a 
distinct name (Fl, F2, F3, etc.) to produce a file called Faces.'" See col. 11, lines 22-26. The 
only discussion possibly related to recording orientation in conjunction with the image is in the 
Examples of 6 and 7, which are "fijn contrast to the system described above [Example 5J. " See 
col. 12, lines 13-15. Therefore, the Examiner's position from the interview improperly combines 
two embodiments that should not be combined. 

Second, if the embodiments were combined, there is no reason that the names Fl, F2, etc. 
rise to the level of an orientation index. The names are only names. The names are not an 
orientation index that identifies the physical orientation of the oriented view of the preregistered 
picture associated with the corresponding orientation index, as required by claim 12, the 
Examiner respectfully disagrees. Bulman discloses that the difference between his examples 5 
and 6 is that in example 6, production is allowed to occur in real time and be synthesized, 
sequenced and directly recorded on a video tape, see Bulman, col 12, lines 14-18. Bulman also 
discloses, in his example 6, that a foreground image is supplied from a library of images in 
memory 102, which may include images varying in perspective, size, action, lighting, or other 
characteristics, see Bulman, col 12, lines 21-25. This, then, reasonably corresponds to the 
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example of Fig. 11. However, even if Bulman's "library of images in memory 102" does not 
specifically refer to Fig. 11, because the images of memory 102 differ in orientation and 
positioning, and that information is recorded in conjunction with the images (see Bulman, col 13, 
lines 29-33), there still reasonably is an index. 

Also, the names Fl, F2, etc., refer to different images, and, if the embodiments were 
combined, would thereby refer to the information that is recorded in conjunction with each of the 
stored images, that are selected based on a cohesive matching of a desired position and 
orientation with the background image (see Bulman, col 13, lines 29-37). Regardless of how the 
images are received, i.e., by photograph, or by the human subject standing on a turntable (see 
Bulman, col 11, lines 1-13 and col 13, lines 37-42), the images are stored and referenced based 
on their orientations and positions. Therefore, the combination of Lemmons in view of Bulman 
does teach the limitations as claimed. 

In response to Applicant's arguments on page 11, lines 8-17 that, "The Office Action 
acknowledges that neither Lemmons /Market nor Bulman describes transmitting each oriented 
view of the first set of oriented views in advance of transmission of the stream of video images. 
See p. 12. Lemmons '981 also fails to describe this feature. 

Lemmons '981 describes a system where a television program is transmitted on one 
channel and enhancements to the television program are transmitted on another channel. See 
abstract. The enhancements may include management messages or program guides. See ^18. 
The receiver may store combined data in a hard drive. See K 29. Lemmons '981 is completely 
silent regarding transmitting oriented views in advance of transmission of the stream of video 
images'", the Applicant should note that one cannot show nonobviousness by attacking references 
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individually where the rejections are based on combinations of references. See In re Keller, 642 
F.2d413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 
(Fed. Cir. 1986). The Applicant should please note that, specifically, the combination of 
Lemmons in view of Bulman and Lemmons '981 was used to reject the above referenced claim, 
since the enhancement information of Lemmons '98 1 that is transmitted in advance of a stream 
of video (see Lemmons '981, [0029]-[0030], lines 1-4) reasonably corresponds to Lemmons in 
view of Bulman's information (see Bulman, Fig. 11. col 12, lines 21-25 and col 13, lines 29-42 
and Lemmons, [0084], lines 5-9, '745, page 9, lines 14-29, and '745, page 12, lines 1-3). 

Claim Rejections - 35 USC §103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

4. Claims 1-3, 7-9, 11-16 and 18 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Lemmons (of record) in view of Bulman (of record) and Wang et al. (US6990681, 
hereinafter Wang). 

Regarding claims 1 and 12, Lemmons discloses a method for generating a stream of 
video images (scenes/frames of the video, see Lemmons, [0031], lines 1-2 and Figs. 6A and 6B) 
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such as, at the reception, in each current video image, a preregistered picture (labels 610/620, see 
Lemmons, Figs. 6A and 6B) can be superimposed to a predetermined area of a moving object 
(soda can 608/618, see Lemmons, Figs. 6A and 6B), comprising: 

determining in each current image a location, orientation and size of said predetermined 
area of said moving object (see Lemmons, [0081]-[0084], line 5), and 

for each current image, using the selected information on the location and size of said 
predetermined area of said moving object to superimpose the preregistered picture on the each 
current image (see Lemmons, [0084], lines 5-9 and '745, page 9, lines 14-29 and page 12, lines 
1-3). 

Lemmons does not specifically disclose providing, with a calculator, a first set of 
oriented views of the preregistered picture in various orientations, 

associating each oriented view of the first set of oriented views with an orientation index 
that identifies the physical orientation of the oriented view, 

storing, in a machine-readable medium, the first set of oriented views associated with 
each orientation index, 

receiving orientation and position data fro the moving object, the orientation and position 
data captured by a sensor attached to the moving object, 

receiving position data for a camera that captured each current image, 

wherein the orientation is determined using the orientation and position data for the 
moving object and the position data for the camera, 



Application/Control Number: 10/696,747 Page 7 

Art Unit: 2421 

selecting, from the orientation indices associated with the stored oriented views, the 
orientation index of the oriented view having the same orientation as said predetermined area of 
said moving object in the current image, or 

transmitting with each current image the selected orientation index. 

In an analogous art relating to a system for producing an electronic image, Bulman 
discloses providing, with a calculator, a first set of oriented views of a preregistered picture in 
various orientations (see Bulman, col 12, lines 21-25 and col 13, lines 29-33), 

associating each oriented view of the first set of oriented views with an orientation index 
that identifies the physical orientation of the oriented view (see Bulman, Fig. 1 1 and col 12, lines 
21-25 and col 13, lines 29-33), 

storing, in a machine-readable medium, the first set of oriented views associated with 
each orientation index (see Bulman, col 12, lines 21-25 and col 13, lines 29-33), 

selecting, from the orientation indices associated with the stored oriented views, the 
orientation index of the oriented view having the same orientation as a predetermined area of an 
object in a current image (see Bulman, col 13, lines 33-37 and Fig. 11.), and 

transmitting with each current image the selected orientation index (The orientation index 
of Bulman's system includes orientation and positioning information along with the different 
orientated views of the picture, see Bulman, col 13, lines 33-37. This then, reasonably 
corresponds to the data file 722 of Lemmons' system, which is transmitted with each image. 
Therefore, Lemmons in view of Bulman reasonably teaches the limitation of transmitting with 
each current image the selected orientation index along with information on the location and size 
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of said predetermined area of said moving object (see Lemmons, [0084], lines 5-9 and '745, page 
9, lines 14-29 and page 12, lines 1-3, and Bulman, col 13, lines 29-33). 

It would have been obvious for a person having ordinary skill in the art at the time of the 
invention to modify Lemmons' system to include the limitations as taught by Bulman for the 
advantage of providing an improved system for allowing a combined image to be displayed with 
a more natural look. 

Lemmons in view of Bulman does not specifically disclose receiving orientation and 
position data fro the moving object, the orientation and position data captured by a sensor 
attached to the moving object, 

receiving position data for a camera that captured each current image, or 

wherein the orientation is determined using the orientation and position data for the 
moving object and the position data for the camera. 

In an analogous art relating to a system for providing enhanced broadcasting, Wang 
discloses receiving orientation and position data from a moving object (see Wang, col 7, lines 
26-59), the orientation and position data captured by a sensor attached to the moving object (see 
Wang, col 7, lines 55-59), 

receiving position data for a camera that captured each current image (see Wang, col 7, 
lines 8-12 and lines 60-66), and 

wherein orientation is determined using the orientation and position data for the moving 
object and the position data for the camera (see Wang, co 8, lines 12-16). 

It would have been obvious for a person having ordinary skill in the art at the time of the 
invention to modify the system of Lemmons in view of Bulman to include the limitations as 
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taught by Wang for the advantage of providing an improved system for allowing a combined 
image to be displayed with a more natural look, and further allowing the possibility to 
successfully integrate properly placed, scaled and oriented synthetic content with video content. 

Regarding claims 2 and 13, Lemmons in view of Bulman and Wang discloses providing 
(see Lemmons, [0084], lines 5-9 and '745, page 9, lines 14-29 and page 12, lines 1-3) at least 
one second set of views of a second preregistered picture (the second set of views are the updates 
to the hot spots/labels, see Lemmons, [0010], lines 10-18, [0013], lines 6-14, [0040], lines 6-7 
and Figs. 6A and 6B), and therefore would correspond to updating the views of the preregistered 
picture of Bulman, see Bulman, col 12, lines 21-25 and col 13, lines 29-33), corresponding to the 
first set of oriented views (the second set of views are the updates to the hot spots/labels, see 
Lemmons, [0010], lines 10-18, [0013], lines 6-14, [0040], lines 6-7 and Figs. 6A and 6B, and 
therefore would correspond to updating the views of the preregistered picture of Bulman, see 
Bulman, col 12, lines 21-25 and col 13, lines 29-33), and for each transmitted current image: 

extracting the orientation index and the size and location information (see Bulman, col 
13, lines 33-37), 

selecting, from the second set of views (the second set of views are the updates to the hot 
spots/labels, see Lemmons, [0010], lines 10-18, [0013], lines 6-14, [0040], lines 6-7 and Figs. 6A 
and 6B, and therefore would correspond to updating the views of the preregistered picture of 
Bulman, see Bulman, col 12, lines 21-25 and col 13, lines 29-33), an oriented picture in 
accordance with the orientation index (see Bulman, col 13, lines 33-37 and Fig. 1 1), 

computing a scaled picture on the basis of the size information (see Lemmons, [0082]- 
[0084], line 5, Figs. 6A-6B and '745, Fig. 7 and page 5, lines 1-9), and 
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superimposing the scaled picture in the current image at a location corresponding to the 
location information (see Lemmons, Figs. 6A-6B and [0082]-[0083], line 5, and '745, Fig. 7 and 
page 5, lines 1-9). 

Regarding claim 3, Lemmons in view of Bulman and Wang discloses, at the beginning of 
a TV program to be transmitted, the second set of views (the second set of views are the updates 
to the hot spots/labels, see Lemmons, [0010], lines 10-18, [0013], lines 6-14 and [0040], lines 6- 
7, and therefore would correspond to updating the views of the preregistered picture of Bulman, 
see Bulman, col 12, lines 21-25 and col 13, lines 29-33) is downloaded in video receivers (As the 
labels/label information may be downloaded, and it is very well known in the art and common 
that a download can occur at any time, Lemmons fairly suggests that at the beginning of a TV 
program to be transmitted, the second set of views is downloaded in video receivers, see 
Lemmons, [0084], lines 5-9 and '745, page 9, lines 14-29 and page 12, lines 1-3). 

Regarding claim 7, Lemmons in view of Bulman and Wang discloses the content of the 
second set of views depends upon the geographic broadcasting zone (see Lemmons, [0013], lines 
6-14 and [0040], lines 1-13). 

Regarding claim 8, Lemmons in view of Bulman and Wang discloses the location and 
orientation information in a current image are calculated for a reference point of the object (see 
Lemmons, [0083]-[0084], line 5). 

Regarding claim 9, Lemmons in view of Bulman and Wang discloses, in a current image, 
the location, orientation and size of an object are provided in a differential way with respect to a 
former image (see '745, page 13, line 30 - page 14, line 11). 
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Regarding claim 1 1 , Lemmons in view of Bulman and Wang discloses using shape 
recognition tools to detect the presence of the moving object in the current image on the basis on 
a stored geometrical representation (see Lemmons, [0064], lines 20-32). 

Regarding claim 14, Lemmons in view of Bulman and Wang discloses a second set of 
views contains picture frames of same orientation of the first set of views, with a picture content 
(the second set of views are the updates to the hot spots, see Lemmons, [0010], lines 10-18, 
[0013], lines 6-14 and [0040], lines 6-7 and Figs. 6A-6B). 

Regarding claim 15, Lemmons in view of Bulman and Wang discloses in which the first 
set of oriented views contains only picture frames (because Lemmons teaches that an object may 
be defined with a hot spot of which may only include an outline of a specific area, it is fairly 
suggested that a first set of views contains only picture frames, as there is no specific ad or label 
visible, see '745, page 5, lines 4-24). 

Regarding claims 16 and 18, Lemmons in view of Bulman discloses superimposing, with 
a video receiver, the oriented view having the same orientation as said predetermined area in the 
current image in the current image (see Lemmons, Figs. 6A-6B and [0082]-[0083], line 5, and 
'745, Fig. 7 and page 5, lines 1-9 and Bulman, col 13, lines 33-37). 

5. Claim 10 is rejected under 35 U.S.C. 103(a) as being unpatentable over Lemmons (of 
record) in view of Bulman (of record) and Wang (previously cited), as applied to claim 1 above, 
and further in view of Wixson (of record). 
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Regarding claim 10, Lemmons in view of Bulman and Wang does not specifically 
disclose static points of an image are localizable to detect when a new object comes into a next 
image. 

In an analogous art relating to a system for detecting and tracking objects, Wixson 
discloses static points of an image are localizable to detect when a new object comes into a next 
image (see col 1, lines 26-32). 

It would have been obvious for a person having ordinary skill in the art at the time of the 
invention to modify the system of Lemmons in view of Bulman and Wang to include the 
limitations as taught by Wixson, for the advantage of providing an advertisement to the newly 
detected object. 

6. Claim 17 is rejected under 35 U.S.C. 103(a) as being unpatentable over Lemmons (of 
record) in view of Bulman (of record) and Wang (previously cited), as applied to claim 1 above, 
and further in view of Martinolich (of record). 

Regarding claim 17, Lemmons in view of Bulman and Wang does not specifically 
disclose superimposing, with a video production mixer, the oriented view having the same 
orientation as the area in the current image in the current image. 

In an analogous art relating to a system for the production of interactive video, 
Martinolich discloses superimposing, with a video production mixer, the oriented view having 
the same orientation as the area in the current image in the current image (see Martinolich , 
[0020], lines 1-8, [0022], lines 1-19 and Fig. 1). 



Application/Control Number: 10/696,747 Page 13 

Art Unit: 2421 

It would have been obvious for a person having ordinary skill in the art at the time of the 
invention to modify the system of Lemmons in view of Bulman and Wang to include the 
limitations as disclosed by Martinolich, for the advantage of using a commonly know and widely 
used device that allows secondary information to be added to a video signal. 

7. Claim 20 is rejected under 35 U.S.C. 103(a) as being unpatentable over Lemmons (of 
record) in view of Bulman (of record) and Lemmons '981 (of record). 

Regarding claim 20, Lemmons discloses a method for transmitting a stream of video 
images (scenes/frames of the video, see Lemmons, [0031], lines 1-2 and Figs. 6A and 6B) such 
that a preregistered picture (labels 610/620, see Lemmons, Figs. 6A and 6B) can be 
superimposed to a predetermined area of a moving object depicted in the stream (soda can 
608/618, see Lemmons, Figs. 6A and 6B), the method comprising: 

for each video image of the stream of video images: 

determining location, orientation and size of the predetermined area of the moving object 
in the video image (see Lemmons, [0083]-[0084], line 5); 

transmitting the video image along with the determined location and the size of the 
predetermined area of the moving object in the video image (see Lemmons, [0084], lines 5-9 and 
'745, page 9, lines 14-29 and page 12, lines 1-3). 

Lemmons does not specifically disclose providing, with a calculator, a first set of 
oriented views of the preregistered picture in various orientations; 
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in advance of transmission of the stream of video images, transmitting each oriented view 
of the first set of oriented views in association with an orientation index that identifies a physical 
orientation of the oriented view of the preregistered picture; 

selecting, from the orientation indices associated with the first set of oriented views, 

an orientation index corresponding to an orientation of the predetermined area of the 
moving object in the video image, or 

transmitting with each current image the selected orientation index. 

In an analogous art, Bulman discloses providing, with a calculator, a first set of oriented 
views of the preregistered picture in various orientations (see Bulman, col 12, lines 21-25 and col 
13, lines 29-33); 

selecting, from the orientation indices associated with the stored oriented views, an 
orientation index of the oriented view having the same orientation as a predetermined area of an 
object in a current image (see Bulman, col 13, lines 33-37 and Fig. 1 1 .), and 

transmitting with each current image the selected orientation index (The orientation index 
of Bulman's system includes orientation and positioning information along with the different 
orientated views of the picture, see Bulman, col 13, lines 33-37. This then, reasonably 
corresponds to the data file 722 of Lemmons' system, which is transmitted with each image. 
Therefore, Lemmons in view of Bulman reasonably teaches the limitation of transmitting with 
each current image the selected orientation index along with information on the location and size 
of said predetermined area of said moving object (see Lemmons, [0084], lines 5-9 and '745, page 
9, lines 14-29 and page 12, lines 1-3, and Bulman, col 13, lines 29-33). 
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It would have been obvious for a person having ordinary skill in the art at the time of the 
invention to modify Lemmons' system to include the limitations as taught by Bulman for the 
advantage of providing an improved system for allowing a combined image to be displayed with 
a more natural look. 

Lemmons in view of Bulman also discloses associating each oriented view of the first set 
of oriented views (i.e., enhancement information) with an orientation index that identifies the 
physical orientation of the oriented view (see Bulman, Fig. 1 1 and col 12, lines 21-25 and col 13, 
lines 29-33), and storing this enhancement information, but does not specifically disclose 
transmitting the enhancement information in advance of transmission of the stream of video 
images. 

In an analogous art, Lemmons '981 discloses transmitting enhancement information in 
advance of transmission of a stream of video images (see [0029]-[0030], lines 1-4). 

It would have been obvious for a person having ordinary skill in the art at the time of the 
invention to modify the system of Lemmons in view of Bulman to include the limitations as 
taught by Lemmons '981 for the advantage of providing an improved system for delivering 
enhancement data while conserving bandwidth. 

8. Claim 21 is rejected under 35 U.S.C. 103(a) as being unpatentable over Lemmons (of 
record) in view of Bulman (of record) and Lemmons '981 (of record), as applied to claim 20 
above, and further in view of Wang (previously cited). 
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Regarding claim 21, Lemmons in view of Bulman and Lemmons '981 does not 
specifically disclose transmitting a polygon representation of an obstruction with the video image 
and the selected orientation index. 

In an analogous art, Wang discloses transmitting a polygon representation of an 
obstruction with the video image and the selected orientation index (Wang discloses creating a 
view of a synthetic camera based on the field of view of an actual camera [see Wang, col 7, lines 
37-39]. Therefore, any object in the camera's field of view, including an obstruction, will be 
represented in the transmitted signal [see Wang, col 7, lines 37-54]). 

It would have been obvious for a person having ordinary skill in the art at the time of the 
invention to modify the system of Lemmons in view of Bulman and Lemmons '981 to include 
the limitations as taught by Wang for the advantage of providing an improved system for 
allowing a combined image to be displayed with a more natural look, and further allowing the 
possibility to successfully integrate properly placed, scaled and oriented synthetic content with 
video content. 

Conclusion 

9. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
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MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to CHENEA P. SMITH whose telephone number is (571)272-9524. 
The examiner can normally be reached on Monday through Friday, 7:30 am - 5:00 pm, EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Miller can be reached on (571) 272-7353. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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